Learning Objectives

After completing this lesson, you’ll be able to:

• Define the FME Flow Resource file system.
• Explain how to add network drives or Amazon S3 buckets as resources.
• Explain the difference between repositories and resources.
• Summarize each data publishing method and evaluate when they are best used.

Video

What are Resources?

The final and preferred method of managing source data in FME Flow is to use Resources.

"Resources" is an inbuilt file management system that allows data (and other files) to be published to an FME Flow instance and used within all FME Flow operations. Resources can be managed through the Resources page accessed from the main menu.

The Resources page is where you can find all of the files that are uploaded to FME Flow and also the files that are created by FME Flow:

As you can see, the Resources file system is set up with many default folders in which files can be stored. Custom folders that you create can also be added to the file system, we'll do this in the next exercise. For datasets the most logical folder to use is the Data folder:

Above is the Data folder containing several files. Once a dataset has been checked, the Actions button is no longer grayed out and lists various actions that can be performed on the file or folder, such as duplicate, edit, upload, copy, delete, or move files (or folders).

Other Upload Methods

Besides the web interface, there are other ways of getting data into the Resources file system.

FME Flow Publishing Wizard

First is the FME Flow publishing wizard in FME Workbench. If not otherwise specified at installation, the default method is to select the files and upload them to the same repository as the workspace. This is the method covered in previous lessons in this course. However, it is preferable to change the location to the Resources file system:

Server File System

Alternatively, FME Flow resources actually exist on the server machine's file system, meaning the data can be copied there directly. The default location (on a Windows operating system) is C:\ProgramData\Safe Software\FMEFlow\resources.

FME Flow Resources Page

A final way to upload data to the Resource system is to click the New button on the main Resources page in FME Flow. This button allows a connection to be made directly to a network-based resource via a UNC path or an Amazon S3 file system (for upload or download).

Some important limitations include that while network-based resources can be used within workspaces, Amazon S3 file systems cannot. Additionally, the resources must be shared with the FME Flow Service Account to use them with FME Flow Services.

Using Uploaded Data

Using Resources data in a translation is simply a case of selecting it from that folder where prompted. All prompts for data allow the selection of files from a Resources folder.

For example, a user has uploaded a MapInfo TAB Parks dataset to the Resources data folder. Provided the source dataset is a published parameter, when the workspace is run the user is able to select data from the Resources folders, like so:

In fact, it's even possible to set the output data folder to be a Resources folder too:

Don't be concerned that the value doesn't show the full file path but rather a value contained within $(). When a folder is stored in Resources, it is assigned an FME Flow User Parameter value, which is essentially a shortcut to the FME Flow System Share file path.

Benefits for Data Management

There are several benefits to using the Resources file system as a data storage tool:

• Data can be used by any workspace, without having to upload it every time.
• Data can be stored locally (to the FME Flow Engines), even when access to the operating system's file system is restricted by an FME Flow Administrator.
• A Resources folder can be mapped and shared among many users as a physical drive.
• A Resources folder is a more permanent solution than temporary uploads. Data is not removed by automated system cleaning tools.